Multi-channel opportunistic access : a restless multi-armed bandit perspective. (Accès opportuniste dans les systèmes de communication multi-canaux : une perspective du problème de bandit-manchot)

Cognitive radio, first envisioned by Mitola, is the key enabling technology for future generations of wireless systems that addresses critical challenges in spectrum efficiency, interference management, and coexistence of heterogeneous networks. The core concept in cognitive radio networks is opportunistic spectrum access, whose objective is to solve the imbalance between spectrum scarcity and spectrum under-utilization. In the thesis, we address the fundamental problem of opportunistic spectrum access in a multi-channel communication system. Specifically, we consider a communication system in which a user has access to multiple channels, but is limited to sensing and transmitting only on part of them at a given time. We explore how the smart user should exploit past observations and the knowledge of the stochastic properties of these channels to maximize its transmission rate by switching channels opportunistically. Formally, we provide a generic analysis on the opportunistic spectrum access problem by casting the problem into the restless multi-armed bandit (RMAB) problem, one of the most well-known generalizations of the classic multi-armed bandit (MAB) problem, which is of fundamental importance in stochastic decision theory. Despite the significant research efforts in the field, the RMAB problem in its generic form still remains open. Until today, very little result is reported on the structure of the optimal policy. Obtaining the optimal policy for a general RMAB problem is often intractable due to the exponential computation complexity. Hence, a natural alternative is to seek a simple myopic policy maximizing the short-term reward. We start by conducting a generic analysis in Chapter 3 on the optimality of the myopic sensing policy where the user senses more than one channel each time and gets one unit of reward if at least one of the sensed channels is in the good state. Through mathematical analysis, we show that the myopic sensing policy is optimal only for a small subset of cases where the user is allowed to sense two channels each slot. In the general case, we give counterexamples to v te l-0 08 32 56 9, v er si on 1 11 J un 2 01 3